Transformation mode encoding and decoding method and apparatus

ABSTRACT

Embodiments of the present invention disclose a transformation mode encoding and decoding method and apparatus. A correlation between the prediction mode and the transformation mode is used, and an optimal transformation mode candidate set is simplified, thereby saving resources when an index of an optimal transformation mode is encoded in encoding header information. In addition, encoding efficiency is higher when an encoding end elects a transformation mode.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2012/084082, filed on Nov. 5, 2012, which claims priority toChinese Patent Application No. 201110346062.4, filed on Nov. 4, 2011,both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

The present invention relates to the field of communicationstechnologies, and in particular, to a transformation mode encoding anddecoding method and apparatus.

BACKGROUND

Transform encoding is a component of a video encoding and decodingtechnology. The transform encoding refers to performing a certainfunctional transformation on a signal to transform the signal in onesignal space to a signal in another signal space, and then quantizingand encoding the transformed signal. Data compression in a transformencoding system has three steps: transforming, quantizing, and entropyencoding. Generally, a transformation process is reversible, that is, aninverse transformation exists, so as to restore original data. Anencoding end obtains residual data of a video image by using predictiveencoding, quantizes the residual data, and encodes the quantizedresidual data in a code stream by entropy encoding, thereby completingan encoding process. A decoding end parses the code stream to obtaindata, reconstructs the residual data by inverse quantizing and inversetransformation, and reconstructs a decoded image with reference to apredicted value obtained from the predictive encoding, therebycompleting a decoding process.

In a current transform encoding technology, different transformationtypes exist according to different transformation matrices, where oneclassification manner is classification, according to a shape of atransformation matrix, into three transformation 1 modes including asquare block transformation, a latitudinal rectangular blocktransformation, and a longitudinal rectangular block transformation, soas to process three residual data block types including a square block,a latitudinal rectangular block, and a longitudinal rectangular blockrespectively, as shown in FIG. 1.

Currently, at a video encoding end, when a transformation mode isselected, an optimal transformation mode is selected from the foregoingthree transformation modes according to a rate distortion optimizationrule; and residual data is transformed by using the selectedtransformation mode, and selection information of the transformationmode is encoded in a code stream and transferred to a decoding end.

At the video encoding end, a mode needs to be selected among threescanning modes, and therefore, complexity at the encoding end is high.In addition, the selection information of the transformation mode needsto be encoded into the code stream, thereby affecting compression andencoding efficiency.

SUMMARY

Embodiments of the present invention provide a transformation modeencoding and decoding method and apparatus, so as to reduce complexityat an encoding end and improve compression and encoding efficiency.

An embodiment of the present invention provides a transformation modeencoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a residual data block of a to-be-encoded video        image;    -   selecting a transformation mode corresponding to the prediction        mode from a candidate transformation mode set; and    -   performing a transformation operation on the residual data block        of the to-be-encoded video image by using the transformation        mode.

An embodiment of the present invention provides a transformation modedecoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image;

selecting a transformation mode corresponding to the prediction modefrom a candidate transformation mode set; and

-   -   performing an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the transformation mode.

An embodiment of the present invention provides a transformation modeencoding apparatus, including:

-   -   a first obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a residual data        block of the to-be-encoded video image;    -   a first selecting unit, configured to select a transformation        mode corresponding to the prediction mode from a candidate        transformation mode set; and    -   a first processing unit, configured to perform a transformation        operation on the residual data block of the to-be-encoded video        image by using the transformation mode.

An embodiment of the present invention provides a transformation modedecoding apparatus, including:

-   -   a fourth obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a transformation        coefficient block of a to-be-decoded image;    -   a fourth selecting unit, configured to select a transformation        mode corresponding to the prediction mode from a candidate        transformation mode set; and    -   a fourth processing unit, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

An embodiment of the present invention provides a transformation modeencoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image;    -   determining a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode;    -   preferentially selecting an optimal transformation mode from the        simplified candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image;    -   performing a transformation operation on the residual data block        of the to-be-encoded video image by using the optimal        transformation mode; and    -   encoding an index of the optimal transformation mode in a code        stream.

An embodiment of the present invention provides a transformation modedecoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image;    -   determining a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode;    -   parsing a code stream to obtain an index of an optimal        transformation mode; and    -   performing an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

An embodiment of the present invention provides a transformation modeencoding apparatus, including:

-   -   a second obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a residual data        block of the to-be-encoded video image;    -   a second determining unit, configured to determine a simplified        candidate transformation mode set according to the prediction        mode and a correspondence between a candidate transformation        mode set and the prediction mode;    -   a second selecting unit, configured to preferentially select an        optimal transformation mode from the simplified candidate        transformation mode set, where the optimal transformation mode        is used to minimize distortion of the to-be-encoded video image        and a reconstructed video image that is obtained by a decoding        end and minimize a quantity of encoding bits required for the        to-be-encoded video image;    -   a second encoding unit, configured to encode an index of the        optimal transformation mode in a code stream; and    -   a second processing unit, configured to perform a transformation        operation on the residual data block of the to-be-encoded video        image by using the optimal transformation mode.

An embodiment of the present invention provides a transformation modedecoding apparatus, including:

-   -   a fifth obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a transformation        coefficient block of a to-be-decoded image;    -   a fifth determining unit, configured to determine a simplified        candidate transformation mode set according to the prediction        mode and a correspondence between a candidate transformation        mode set and the prediction mode;    -   a fifth decoding unit, configured to parse a code stream to        obtain an index of an optimal transformation mode; and    -   a fifth processing unit, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

An embodiment of the present invention provides a transformation modeencoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image;    -   preferentially selecting an optimal transformation mode from a        candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image;    -   determining whether the optimal transformation mode is a mode        having a correspondence with the prediction mode;    -   writing a determination result in a code stream;    -   if the determination result is no, encoding an index of the        optimal transformation mode in the code stream; and    -   performing a transformation operation on the residual data block        of the to-be-encoded video image by using the optimal        transformation mode.

An embodiment of the present invention provides a transformation modedecoding method, including:

-   -   obtaining a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image;    -   parsing a code stream to obtain a determination result about        whether an optimal transformation mode is a mode having a        correspondence with the prediction mode;    -   if the determination result is no, parsing the code stream to        obtain an index of the optimal transformation mode; and    -   performing an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

An embodiment of the present invention provides a transformation modeencoding apparatus, including:

-   -   a third obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a residual data        block of the to-be-encoded video image; a third selecting unit,        configured to preferentially select an optimal transformation        mode from a candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image;    -   a third determining unit, configured to determine whether the        optimal transformation mode is a mode having a correspondence        with the prediction mode;    -   a third encoding unit, configured to encode the determination        result and an index of the optimal transformation mode in a code        stream; and    -   a third processing unit, configured to perform a transformation        operation on the residual data block of the to-be-encoded video        image by using the optimal transformation mode.

An embodiment of the present invention provides a transformation modedecoding apparatus, including:

-   -   a sixth obtaining unit, configured to obtain a prediction mode        of a predicted data block that corresponds to a transformation        coefficient block of a to-be-decoded image;    -   a sixth determining unit, configured to determine a        transformation mode that corresponds to the prediction mode and        is in a candidate transformation mode set;    -   a sixth decoding unit, configured to parse a code stream to        obtain a determination result about whether an optimal        transformation mode is a mode having a correspondence with the        prediction mode, and an index of the optimal transformation        mode; and    -   a sixth processing unit, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

By using the foregoing technical solutions, in the embodiments of thepresent invention, a correlation between the prediction mode and thetransformation mode is used, and an optimal transformation modecandidate set is simplified, thereby saving resources when an index ofan optimal transformation mode is encoded in encoding headerinformation. In addition, encoding efficiency is higher when an encodingend elects a transformation mode.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the embodiments of the presentinvention more clearly, the following briefly introduces accompanyingdrawings required for describing the embodiments. Apparently, theaccompanying drawings in the following description are only someembodiments of the present invention, and a person of ordinary skill inthe art may still derive other drawings from the accompanying drawingswithout creative efforts.

FIG. 1A is a schematic diagram of shapes of a residual data block of theto-be-encoded video image and a transformation coefficient block of ato-be-decoded image that correspond to a square block transformationaccording to an embodiment of the present invention;

FIG. 1B is a schematic diagram of shapes of a residual data block of theto-be-encoded video image and a transformation coefficient block of ato-be-decoded image that correspond to a latitudinal rectangular blocktransformation according to an embodiment of the present invention;

FIG. 1C is a schematic diagram of shapes of a residual data block of theto-be-encoded video image and a transformation coefficient block of ato-be-decoded image that correspond to a longitudinal rectangular blocktransformation according to an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 3 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 4 is a schematic structural diagram of an apparatus according to anembodiment of the present invention;

FIG. 5 is a schematic structural diagram of an apparatus according to anembodiment of the present invention;

FIG. 6 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 7 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 8 is a schematic structural diagram of an apparatus according to anembodiment of the present invention;

FIG. 9 is a schematic structural diagram of an apparatus according to anembodiment of the present invention;

FIG. 10 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 11 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 12 is a schematic structural diagram of an apparatus according toan embodiment of the present invention;

FIG. 13 is a schematic structural diagram of an apparatus according toan embodiment of the present invention;

FIG. 14 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 15 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 16 is a schematic structural diagram of an apparatus according toan embodiment of the present invention;

FIG. 17 is a schematic structural diagram of an apparatus according toan embodiment of the present invention;

FIG. 18 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 19 is a schematic flowchart of a method according to an embodimentof the present invention;

FIG. 20 is a schematic structural diagram of an apparatus according toan embodiment of the present invention; and

FIG. 21 is a schematic structural diagram of an apparatus according toan embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

The following clearly and completely describes the technical solutionsin the embodiments of the present invention with reference to theaccompanying drawings in the embodiments of the present invention.Apparently, the embodiments to be described are only a part rather thanall of the embodiments of the present invention. All other embodimentsobtained by a person of ordinary skill in the art based on theembodiments of the present invention without creative efforts shall fallwithin the protection scope of the present invention.

In the embodiments of the present invention, during an encoding process,an encoding end selects, for each to-be-encoded block, an optimalencoding type from various encoding types according to a rate distortionrule; obtains prediction information; obtains a predicted value of theto-be-encoded block according to the prediction information andinformation of an encoded reconstructed frame; calculates a differencebetween the predicted value and a pixel value of the to-be-encodedblock, so as to obtain a residual value; sequentially performsoperations of transformation, quantization, scanning, and entropyencoding on the residual value, and encodes a value obtained after theoperations in a code stream; and encodes the prediction information inthe code stream, thereby completing the encoding process. The followingdescribes the scanning in the encoding process and a decoding process indetail. The transformation mentioned in the embodiments of the presentinvention may be a discrete cosine transformation, or may also be amulti-transformation technology, which is not limited in the embodimentsof the present invention.

Exemplary solution 1: The following describes a transformation modeselection procedure executed at an encoding end in detail. As shown inFIG. 2, the transformation mode selection procedure includes:

-   -   101: Obtain a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image, and more specifically, obtain an intra-frame prediction        mode of the predicted data block that corresponds to the        residual data block of the to-be-encoded video image.    -   102: Select a transformation mode corresponding to the        prediction mode from a candidate transformation mode set, and        more specifically, determine a correspondence between the        prediction mode and the transformation mode according to a        principle that “if the prediction mode is an intra-frame        prediction mode, when a quantity of pixels that are among        predicted pixels in the prediction mode and are located at an        upper boundary and an upper right boundary of an encoding unit        exceeds a set quantity, select a latitudinal rectangular block        transformation as a transformation type corresponding to the        prediction mode; when a quantity of pixels that are among the        predicted pixels in the prediction mode and are located at a        left boundary and a lower left boundary of the encoding unit        exceeds the set quantity, select a longitudinal rectangular        block transformation as a transformation type corresponding to        the prediction mode; otherwise, select a square block        transformation as a transformation type corresponding to the        prediction mode”. For example, in a horizontal prediction mode,        select the longitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode, and in        a vertical prediction mode, select the latitudinal rectangular        block transformation as a transformation type corresponding to        the prediction mode. A specific correspondence is not        specifically limited in the principle. The quantity of the        pixels among the predicted pixels and with a quantity exceeding        the set quantity may be all, half, or two thirds, which is not        specifically limited herein.    -   103: Perform a transformation operation on the residual data        block of the to-be-encoded video image by using the        transformation mode.

Correspondingly, an embodiment of the present invention further providesa transformation mode selection procedure executed at a decoding end. Asshown in FIG. 3, the transformation mode selection procedure includes:

-   -   201: Obtain a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image, and more specifically, obtain an        intra-frame prediction mode of the predicted data block that        corresponds to the transformation coefficient block of the        to-be-decoded image.    -   202: Select a transformation mode corresponding to the        prediction mode from a candidate transformation mode set        according to a rule that is the same as that in 102.    -   203: Perform an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the transformation mode.

A transformation mode encoding apparatus, as shown in FIG. 4, includes:

-   -   a first obtaining unit 301, configured to obtain a prediction        mode of a predicted data block that corresponds to a residual        data block of the to-be-encoded video image;    -   a first selecting unit 302, configured to select a        transformation mode corresponding to the prediction mode from a        candidate transformation mode set; and    -   a first processing unit 303, configured to perform a        transformation operation on the residual data block of the        to-be-encoded video image by using the transformation mode.

A transformation mode decoding apparatus, as shown in FIG. 5, includes:

-   -   a fourth obtaining unit 401, configured to obtain a prediction        mode of a predicted data block that corresponds to a        transformation coefficient block of a to-be-decoded image;    -   a fourth selecting unit 402, configured to select a        transformation mode corresponding to the prediction mode from a        candidate transformation mode set; and    -   a fourth processing unit 403, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

Exemplary solution 2: The following describes a transformation modeselection procedure executed at an encoding end in detail. As shown inFIG. 6, the transformation mode selection procedure includes:

-   -   501: Obtain a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image, and more specifically, obtain an intra-frame prediction        mode of the predicted data block that corresponds to the        residual data block of the to-be-encoded video image.    -   502: Determine a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode, and        more specifically, determine the simplified candidate        transformation mode set according to a principle that “if a        quantity of pixels that are among predicted pixels in the        prediction mode and are located at an upper boundary and an        upper right boundary of an encoding unit exceeds a set quantity,        the candidate transformation mode is a latitudinal rectangular        block transformation and a square block transformation; if a        quantity of pixels that are among the predicted pixels in the        prediction mode and are located at a left boundary and a lower        left boundary of the encoding unit exceeds the set quantity, the        candidate transformation mode is a longitudinal rectangular        block transformation and the square block transformation;        otherwise, the candidate transformation mode is the square block        transformation”. For example, in a horizontal prediction mode,        select the longitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode and the        square block transformation, and in a vertical prediction mode,        select the latitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode and the        square block transformation. A specific correspondence is not        specifically limited in the principle. The quantity of the        pixels among the predicted pixels and with a quantity exceeding        the set quantity may be all, half, or two thirds, which is not        specifically limited herein.    -   503: Preferentially select an optimal transformation mode from        the simplified candidate transformation mode set, where the        optimal transformation mode is used to minimize distortion of        the to-be-encoded video image and a reconstructed video image        that is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image, and a        method such as rate distortion optimization may be used, but a        specific method is not limited herein.    -   504: Perform a transformation operation on the residual data        block of the to-be-encoded video image by using the        transformation mode.    -   505: Encode an index of the optimal transformation mode in a        code stream, and more specifically, when the candidate        transformation mode is the latitudinal rectangular block and the        square block, use a flag bit 1 for indication; when the        candidate transformation mode is the longitudinal rectangular        block and the square block, use the flag bit 1 for indication;

11 and when the candidate transformation mode is the square block, noflag bit needs to be used for indication. Step 504 and step 505 are notin a sequential relationship.

Correspondingly, an embodiment of the present invention further providesa transformation mode selection procedure executed at a decoding end. Asshown in FIG. 7, the transformation mode selection procedure includes:

-   -   601: Obtain a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image, and more specifically, obtain an        intra-frame prediction mode of the predicted data block that        corresponds to the transformation coefficient block of the        to-be-decoded image.    -   602: Determine a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode and        according to a rule that is the same as that in 502.    -   603: Parse a code stream to obtain an index of an optimal        transformation mode, and more specifically, corresponding to        503, when a candidate transformation mode is a latitudinal        rectangular block and a square block, parse a flag bit 1 to        obtain the index; when the candidate transformation mode is a        longitudinal rectangular block and the square block, parse the        flag bit 1 to obtain the index; and particularly, when the        candidate transformation mode is the square block, no flag bit        needs to be parsed.    -   604: Perform an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

A transformation mode encoding apparatus, as shown in FIG. 8, includes:

-   -   a second obtaining unit 701, configured to obtain a prediction        mode of a predicted data block that corresponds to a residual        data block of the to-be-encoded video image;    -   a second determining unit 702, configured to determine a        simplified candidate transformation mode set according to the        prediction mode and a correspondence between a candidate        transformation mode set and the prediction mode;    -   a second selecting unit 703, configured to preferentially select        an optimal transformation mode from the simplified candidate        transformation mode set, where the optimal transformation mode        is used to minimize distortion of the to-be-encoded video image        and a reconstructed video image that is obtained by a decoding        end and minimize a quantity of encoding bits required for the        to-be-encoded video image;    -   a second encoding unit 704, configured to encode an index of the        optimal transformation mode in a code stream; and    -   a second processing unit 705, configured to perform a        transformation operation on the residual data block of the        to-be-encoded video image by using the optimal transformation        mode.

A transformation mode decoding apparatus, as shown in FIG. 9, includes:

-   -   a fifth obtaining unit 801, configured to obtain a prediction        mode of a predicted data block that corresponds to a        transformation coefficient block of a to-be-decoded image;    -   a fifth determining unit 802, configured to determine a        simplified candidate transformation mode set according to the        prediction mode and a correspondence between a candidate        transformation mode set and the prediction mode;    -   a fifth decoding unit 803, configured to parse a code stream to        obtain an index of an optimal transformation mode; and    -   a fifth processing unit 804, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

Exemplary solution 3: The following describes a transformation modeselection procedure executed at an encoding end in detail. As shown inFIG. 10, the transformation mode selection procedure includes:

-   -   901: Obtain a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image, and more specifically, obtain an intra-frame prediction        mode of the predicted data block that corresponds to the        residual data block of the to-be-encoded video image.    -   902: Determine a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode, and        more specifically, determine the simplified candidate        transformation mode set according to a principle that “if a        quantity of pixels that are among predicted pixels in the        prediction mode and are located at an upper boundary and an        upper right boundary of an encoding unit exceeds a set quantity,        the candidate transformation mode is a latitudinal rectangular        block transformation and a square block transformation; if a        quantity of pixels that are among the predicted pixels in the        prediction mode and are located at a left boundary and a lower        left boundary of the encoding unit exceeds the set quantity, the        candidate transformation mode is a longitudinal rectangular        block transformation and the square block transformation;        otherwise, the candidate transformation mode is the latitudinal        rectangular block transformation, the longitudinal rectangular        block transformation, and the square block transformation”. For        example, in a horizontal prediction mode, select the        longitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode and the        square block transformation, and in a vertical prediction mode,        select the latitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode and the        square block transformation. A specific correspondence is not        specifically limited in the principle. The quantity of the        pixels among the predicted pixels and with a quantity exceeding        the set quantity may be all, half, or two thirds, which is not        specifically limited herein.    -   903: Preferentially select an optimal transformation mode from        the simplified candidate transformation mode set, where the        optimal transformation mode is used to minimize distortion of        the to-be-encoded video image and a reconstructed video image        that is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image, and a        method such as rate distortion optimization may be used, but a        specific method is not limited herein.    -   904: Perform a transformation operation on the residual data        block of the to-be-encoded video image by using the        transformation mode.    -   905: Encode an index of the optimal transformation mode in a        code stream, and more specifically, when the candidate        transformation mode is the latitudinal rectangular block and the        square block, use a flag bit 1 for indication; when the        candidate transformation mode is the longitudinal rectangular        block and the square block, use the flag bit 1 for indication;        and when the candidate transformation mode is the latitudinal        rectangular block, the longitudinal rectangular block, and the        square block, use the flag bit 1 to indicate whether the        candidate transformation mode is the square block, and if the        candidate transformation mode is not the square block, use the        flag bit 1 again to indicate whether the candidate        transformation mode is the latitudinal rectangular block or the        longitudinal rectangular block. Step 904 and step 905 are not in        a sequential relationship.

Correspondingly, an embodiment of the present invention further providesa transformation mode selection procedure executed at a decoding end. Asshown in FIG. 11, the transformation mode selection procedure includes:

-   -   1001: Obtain a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image, and more specifically, obtain an        intra-frame prediction mode of the predicted data block that        corresponds to the transformation coefficient block of the        to-be-decoded image.    -   1002: Determine a simplified candidate transformation mode set        according to the prediction mode and a correspondence between a        candidate transformation mode set and the prediction mode and        according to a rule that is the same as that in 902.    -   1003: Parse a code stream to obtain an index of an optimal        transformation mode, and more specifically, corresponding to        903, when a candidate transformation mode is a latitudinal        rectangular block and a square block, parse a flag bit 1 to        obtain the index; when the candidate transformation mode is a        longitudinal rectangular block and the square block, parse the        flag bit 1 to obtain the index; and when the candidate        transformation mode is the latitudinal rectangular block, the        longitudinal rectangular block, and the square block, parse the        flag bit 1 to obtain whether the candidate transformation mode        is the square block, and if the candidate transformation mode is        not the square block, parse the flag bit 1 again to obtain        whether the candidate transformation mode is the latitudinal        rectangular block or the longitudinal rectangular block.    -   1004: Perform an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

A transformation mode encoding apparatus, as shown in FIG. 12, includes:

-   -   a second obtaining unit 1101, configured to obtain a prediction        mode of a predicted data block that corresponds to a residual        data block of the to-be-encoded video image;    -   a second determining unit 1102, configured to determine a        simplified candidate transformation mode set according to the        prediction mode and a correspondence between a candidate        transformation mode set and the prediction mode;    -   a second selecting unit 1103, configured to preferentially        select an optimal transformation mode from the simplified        candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image;    -   a second encoding unit 1104, configured to encode an index of        the optimal transformation mode in a code stream; and    -   a second processing unit 1105, configured to perform a        transformation operation on the residual data block of the        to-be-encoded video image by using the optimal transformation        mode.

A transformation mode decoding apparatus, as shown in FIG. 13, includes:

-   -   a fifth obtaining unit 1201, configured to obtain a prediction        mode of a predicted data block that corresponds to a        transformation coefficient block of a to-be-decoded image;    -   a fifth determining unit 1202, configured to determine a        simplified candidate transformation mode set according to the        prediction mode and a correspondence between a candidate        transformation mode set and the prediction mode;    -   a fifth decoding unit 1203, configured to parse a code stream to        obtain an index of an optimal transformation mode; and    -   a fifth processing unit 1204, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

Exemplary solution 4: The following describes a transformation modeselection procedure executed at an encoding end in detail. As shown inFIG. 14, the transformation mode selection procedure includes:

-   -   1301: Obtain a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image, and more specifically, obtain an intra-frame prediction        mode of the predicted data block that corresponds to the        residual data block of the to-be-encoded video image.    -   1302: Preferentially select an optimal transformation mode from        a candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image, and a        method such as rate distortion optimization may be used, but a        specific method is not limited herein.    -   1303: According to a principle that “if a quantity of pixels        that are among predicted pixels in the prediction mode and are        located at an upper boundary and an upper right boundary of an        encoding unit exceeds a set quantity, the candidate        transformation mode selects a latitudinal rectangular block        transformation to serve as a corresponding transformation type        of the prediction mode; when a quantity of pixels that are among        the predicted pixels in the prediction mode and are located at a        left boundary and a lower left boundary of the encoding unit        exceeds the set quantity, a longitudinal rectangular block        transformation is selected as the candidate transformation mode        and as a transformation type corresponding to the prediction        mode; otherwise, the candidate transformation mode corresponds        to a square block transformation”, determine whether the optimal        transformation mode is a mode having a correspondence with the        prediction mode. For example, in a horizontal prediction mode,        select the longitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode, and in        a vertical prediction mode, select the latitudinal rectangular        block transformation as a transformation type corresponding to        the prediction mode. A specific correspondence is not        specifically limited in the principle. The quantity of the        pixels among the predicted pixels and with a quantity exceeding        the set quantity may be all, half, or two thirds, which is not        specifically limited herein.    -   1304: Encode a determination result in a code stream by using a        flag bit 1.    -   1305: If the determination is no, encode an index of the optimal        transformation mode in the code stream, and more specifically,        use the flag bit 1 for indication.    -   1306: Perform a transformation operation on the residual data        block of the to-be-encoded video image by using the        transformation mode. Steps 1304 and 1305 and step 1306 are not        in a sequential relationship.

Correspondingly, an embodiment of the present invention further providesa transformation mode selection procedure executed at a decoding end. Asshown in FIG. 15, the transformation mode selection procedure includes:

-   -   1401: Obtain a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image, and more specifically, obtain an        intra-frame prediction mode of the predicted data block that        corresponds to the transformation coefficient block of the        to-be-decoded image.    -   1402: According to a rule that is the same as that in 1303,        determine a transformation mode having a correspondence with the        prediction mode.    -   1403: Corresponding to 1304, parse a flag bit 1 of a code stream        to obtain a determination result about whether an optimal        transformation mode is a mode having a correspondence with the        prediction mode.    -   1404: Corresponding to 1305, if the determination result is no,        parse the flag bit 1 of the code stream to obtain an index of        the optimal transformation mode.    -   1405: Perform an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

A transformation mode encoding apparatus, as shown in FIG. 16, includes:

-   -   a third obtaining unit 1501, configured to obtain a prediction        mode of a predicted data block that corresponds to a residual        data block of the to-be-encoded video image;    -   a third selecting unit 1502, configured to preferentially select        an optimal transformation mode from a candidate transformation        mode set, where the optimal transformation mode is used to        minimize distortion of the to-be-encoded video image and a        reconstructed video image that is obtained by a decoding end and        minimize a quantity of encoding bits required for the        to-be-encoded video image;    -   a third determining unit 1503, configured to determine whether        the optimal transformation mode is a mode having a        correspondence with the prediction mode;    -   a third encoding unit 1504, configured to encode the        determination result and an index of the optimal transformation        mode in a code stream; and    -   a third processing unit 1505, configured to perform a        transformation operation on the residual data block of the        to-be-encoded video image by using the optimal transformation        mode.

A transformation mode decoding apparatus, as shown in FIG. 17, includes:

-   -   a sixth obtaining unit 1601, configured to obtain a prediction        mode of a predicted data block that corresponds to a        transformation coefficient block of a to-be-decoded image;    -   a sixth determining unit 1602, configured to determine, in a        candidate transformation mode set, a transformation mode        corresponding to the prediction mode;    -   a sixth decoding unit 1603, configured to parse a code stream to        obtain a determination result about whether an optimal        transformation mode is a mode having a correspondence with the        prediction mode, and an index of the optimal transformation        mode; and    -   a sixth processing unit 1604, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

Exemplary solution 5: The following describes a transformation modeselection procedure executed at an encoding end in detail. As shown inFIG. 18, the transformation mode selection procedure includes:

-   -   1701: Obtain a prediction mode of a predicted data block that        corresponds to a residual data block of the to-be-encoded video        image, and more specifically, obtain an intra-frame prediction        mode of the predicted data block that corresponds to the        residual data block of the to-be-encoded video image.    -   1702: Preferentially select an optimal transformation mode from        a candidate transformation mode set, where the optimal        transformation mode is used to minimize distortion of the        to-be-encoded video image and a reconstructed video image that        is obtained by a decoding end and minimize a quantity of        encoding bits required for the to-be-encoded video image, and a        method such as rate distortion optimization may be used, but a        specific method is not limited herein.    -   1703: According to a principle that “if a quantity of pixels        that are among predicted pixels in the prediction mode and are        located at an upper boundary and an upper right boundary of an        encoding unit exceeds a set quantity, a latitudinal rectangular        block transformation is selected as the candidate transformation        mode and as a transformation type corresponding to the        prediction mode; when a quantity of pixels that are among the        predicted pixels in the prediction mode and are located at a        left boundary and a lower left boundary of the encoding unit        exceeds the set quantity, a longitudinal rectangular block        transformation is selected as the candidate transformation mode        and as a transformation type corresponding to the prediction        mode; otherwise, the optimal candidate transformation mode is a        square block transformation”, determine whether the optimal        transformation mode is a mode having a correspondence with the        prediction mode. For example, in a horizontal prediction mode,        select the longitudinal rectangular block transformation as a        transformation type corresponding to the prediction mode, and in        a vertical prediction mode, select the latitudinal rectangular        block transformation as a transformation type corresponding to        the prediction mode. A specific correspondence is not        specifically limited in the principle. The quantity of the        pixels among the predicted pixels and with a quantity exceeding        the set quantity may be all, half, or two thirds, which is not        specifically limited herein.    -   1704: Encode a determination result in a code stream by using a        flag bit 1, and particularly, when the correspondence is only a        correspondence with the square block, no flag bit needs to be        encoded.    -   1705: If the determination is no, encode an index of the optimal        transformation mode in the code stream, and more specifically,        use the flag bit 1 for indication.    -   1706: Perform a transformation operation on the residual data        block of the to-be-encoded video image by using the        transformation mode. Steps 1704 and 1705 and step 1706 are not        in a sequential relationship.

Correspondingly, an embodiment of the present invention further providesa transformation mode selection procedure executed at a decoding end. Asshown in FIG. 19, the transformation mode selection procedure includes:

-   -   1801: Obtain a prediction mode of a predicted data block that        corresponds to a transformation coefficient block of a        to-be-decoded image, and more specifically, obtain an        intra-frame prediction mode of the predicted data block that        corresponds to the transformation coefficient block of the        to-be-decoded image.    -   1802: According to a rule that is the same as that in 1703,        determine a transformation mode having a correspondence with the        prediction mode.    -   1803: Corresponding to 1704, parse a flag bit 1 of a code stream        to obtain a determination result about whether an optimal        transformation mode is a mode having a correspondence with the        prediction mode, and particularly, when the correspondence is        only a correspondence with a square block, no flag bit needs to        be parsed.    -   1804: Corresponding to 1705, if the determination result is no,        parse the flag bit 1 of the code stream to obtain an index of        the optimal transformation mode.    -   1805: Perform an inverse transformation operation on the        transformation coefficient block of the to-be-decoded image by        using the optimal transformation mode.

A transformation mode encoding apparatus, as shown in FIG. 20, includes:

-   -   a third obtaining unit 1901, configured to obtain a prediction        mode of a predicted data block that corresponds to a residual        data block of the to-be-encoded video image;    -   a third selecting unit 1902, configured to preferentially select        an optimal transformation mode from a candidate transformation        mode set, where the optimal transformation mode is used to        minimize distortion of the to-be-encoded video image and a        reconstructed video image that is obtained by a decoding end and        minimize a quantity of encoding bits required for the        to-be-encoded video image;    -   a third determining unit 1903, configured to determine whether        the optimal transformation mode is a mode having a        correspondence with the prediction mode;    -   a third encoding unit 1904, configured to encode the        determination result and an index of the optimal transformation        mode in a code stream; and    -   a third processing unit 1905, configured to perform a        transformation operation on the residual data block of the        to-be-encoded video image by using the optimal transformation        mode.

A transformation mode decoding apparatus, as shown in FIG. 21, includes:

-   -   a sixth obtaining unit 2001, configured to obtain a prediction        mode of a predicted data block that corresponds to a        transformation coefficient block of a to-be-decoded image;    -   a sixth determining unit 2002, configured to determine, in a        candidate transformation mode set, a transformation mode        corresponding to the prediction mode; and a sixth decoding unit        2003, configured to parse a code stream to obtain a        determination result about whether an optimal transformation        mode is a mode having a correspondence with the prediction mode,        and an index of the optimal transformation mode; and    -   a sixth processing unit 2004, configured to perform an inverse        transformation operation on the transformation coefficient block        of the to-be-decoded image by using the transformation mode.

In the foregoing different embodiments, a correlation between predictioninformation and transformation mode information is used, and because ofexistence of the correlation, some redundant candidates in a candidatetransformation mode set may be eliminated. In the embodiments, thecandidate transformation mode set is simplified from differentperspectives, encoding efficiency is improved at an encoding end, andencoding time and resource overhead are saved. Moreover, because aquantity of candidate modes is reduced, a smaller quantity of encodingbits may be used to indicate selection information of the candidatemodes, thereby reducing header information and improving encodingperformance.

The method provided in the embodiments of the present invention may beapplied in the field of digital signal processing, and implemented byusing a video encoder and decoder. The video encoder and decoder arewidely applied in various communications devices or electronic devices,such as a media gateway, a mobile phone, a wireless apparatus, apersonal digital assistant (PDA), a handheld or portable computer, a GPSreceiver/navigator, a camera, a video player, a video camera, a videorecorder, and a monitoring device. The devices include a processor, amemory, and an interface for transmitting data. The video encoder anddecoder may be directly implemented by using a digital circuit or chipsuch as a DSP (digital signal processor); or may be implemented bysoftware code driving a processor to execute a process in the softwarecode.

A person of ordinary skill in the art can understand that, all or a partof the steps of the methods in the foregoing embodiments may beimplemented by a program instructing relevant hardware. The foregoingprogram may be stored in a computer readable storage medium. The storagemedium may be a read only memory, a magnetic disk, an optical disc, orthe like.

The foregoing describes a method, an apparatus, and a system forscanning a transformation coefficient block of a to-be-decoded imagethat are provided in the embodiments of the present invention in detail.Specific examples are used in this specification to describe theprinciple and implementation manners of the present invention. Thedescriptions of the foregoing embodiments are merely intended to helpunderstand the method and core idea of the present invention. Inaddition, with respective to the implementation manners and theapplication scope, modifications may be made by a person of ordinaryskill in the art according to the idea of the present invention.Therefore, this specification shall not be construed as a limitation onthe present invention.

1. A transformation mode encoding method, comprising: obtaining aprediction mode of a predicted data block that corresponds to a residualdata block of a video image to be encoded; selecting a transformationmode corresponding to the prediction mode from a candidatetransformation mode set; and performing a transformation operation onthe residual data block of the video image based on the transformationmode.
 2. A transformation mode encoding method, comprising: obtaining aprediction mode of a predicted data block that corresponds to a residualdata block of a video image to be encoded; determining a simplifiedcandidate transformation mode set according to the prediction mode and acorrespondence between a candidate transformation mode set and theprediction mode; selecting a transformation mode from the simplifiedcandidate transformation mode set, wherein the transformation mode isused to minimize distortion of the video image and a reconstructed videoimage that is obtained by a decoding end and to minimize a quantity ofencoding bits required for encoding the video image; performing atransformation operation on the residual data block of the video imagebased on the optimal transformation mode; and encoding an index of theoptimal transformation mode in a code stream.
 3. A transformation modeencoding method, comprising: obtaining a prediction mode of a predicteddata block that corresponds to a residual data block of a video image tobe encoded; selecting a transformation mode from a candidatetransformation mode set, wherein the transformation mode is used tominimize distortion of the video image and a reconstructed video imagethat is obtained by a decoding end and to minimize a quantity ofencoding bits required for the video image; determining whether theoptimal transformation mode is a mode having a correspondence with theprediction mode; writing a determination result in a code stream; if theoptimal transformation mode is not a mode having a correspondence withthe prediction mode, then encoding an index of the optimaltransformation mode in the code stream; and performing a transformationoperation on the residual data block of the video image based on theoptimal transformation mode.
 4. The method according to claim 1, whereinthe prediction mode of the predicted data block that corresponds to theresidual data block of the video image comprises: an intra-frameprediction mode in intra-frame prediction; or an inter-frame predictionunit division mode in inter-frame prediction; or an intra-frameprediction unit division mode in intra-frame prediction; or atransformation mode of a transformation coefficient block of a videoimage in a time domain or a spatial domain of the residual data block ofthe video image.
 5. The method according to claim 1, wherein thecandidate transformation mode set comprises: a square blocktransformation, a latitudinal rectangular block transformation, and alongitudinal rectangular block transformation; or a transformationreflecting no texture and an irregular texture, a transformationreflecting a latitudinal texture, and a transformation reflecting alongitudinal texture; or a transformation reflecting no texture, atransformation reflecting an irregular texture, a transformationreflecting a latitudinal texture, and a transformation reflecting alongitudinal texture.
 6. The method according to claim 1, wherein thecorrespondence between the candidate transformation mode set and theprediction mode comprises: if the prediction mode is an intra-frameprediction mode, when a quantity of pixels that are among predictedpixels in the prediction mode and are located at an upper boundary andan upper right boundary of an encoding unit exceeds a set quantity,selecting a latitudinal rectangular block transformation as atransformation type corresponding to the prediction mode; when aquantity of pixels that are among the predicted pixels in the predictionmode and are located at a left boundary and a lower left boundary of theencoding unit exceeds the set quantity, selecting a longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; otherwise, selecting a square blocktransformation as a transformation type corresponding to the predictionmode; or if the prediction mode is an intra-frame prediction mode, whena quantity of pixels that are among predicted pixels in the predictionmode and are located at an upper boundary and an upper right boundary ofan encoding unit exceeds a set quantity, selecting a non-longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; when a quantity of pixels that are among thepredicted pixels in the prediction mode and are located at a leftboundary and a lower left boundary of the encoding unit exceeds the setquantity, selecting a non-latitudinal rectangular block transformationas a transformation type corresponding to the prediction mode;otherwise, selecting a preset transformation type in all transformationtypes as a transformation type corresponding to the prediction mode; orif prediction information represents a vertical texture, selecting alatitudinal rectangular block transformation as a transformation typecorresponding to the prediction mode; if the prediction informationrepresents a horizontal texture, selecting a longitudinal rectangularblock transformation as a transformation type corresponding to theprediction mode; and if the prediction information represents no textureor an irregular texture, selecting a square block transformation as atransformation type corresponding to the prediction mode; or ifprediction information represents a vertical texture, selecting anon-longitudinal rectangular block transformation as a transformationtype corresponding to the prediction mode; if the prediction informationrepresents a horizontal texture, selecting a non-latitudinal rectangularblock transformation as a transformation type corresponding to theprediction mode; and if the prediction information represents no textureor an irregular texture, corresponding to all types of transformations;or if prediction information represents a latitudinal ornon-longitudinal rectangular block transformation, selecting thelatitudinal rectangular block transformation as a transformation typecorresponding to the prediction mode; if the prediction informationrepresents a longitudinal or non-latitudinal rectangular blocktransformation, selecting the longitudinal rectangular blocktransformation as a transformation type corresponding to the predictionmode; and if the prediction information represents a square blocktransformation, selecting the square block transformation as atransformation type corresponding to the prediction mode; or ifprediction information represents a latitudinal or non-longitudinalrectangular block transformation, selecting the non-longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; if the prediction information represents alongitudinal or non-latitudinal rectangular block transformation,selecting the non-latitudinal rectangular block transformation as atransformation type corresponding to the prediction mode; and if theprediction information represents a square block transformation,corresponding to all types of transformations.
 7. The method accordingto claim 2, wherein the simplified candidate transformation mode set isa set of candidate transformation modes having the correspondence withthe prediction mode.
 8. The method according to claim 7, furthercomprising simplifying the candidate transformation mode set, whereinsimplifying the candidate transformation mode set comprises: if aquantity of pixels that are among predicted pixels in the predictionmode and are located at an upper boundary and an upper right boundary ofan encoding unit exceeds a set quantity, a candidate transformation modeis a latitudinal rectangular block transformation and a square blocktransformation; if a quantity of pixels that are among the predictedpixels in the prediction mode and are located at a left boundary and alower left boundary of the encoding unit exceeds the set quantity, thecandidate transformation mode is a longitudinal rectangular blocktransformation and the square block transformation; otherwise, thecandidate transformation mode is the square block transformation; or ifa quantity of pixels that are among predicted pixels in the predictionmode and are located at an upper boundary and an upper right boundary ofan encoding unit exceeds a set quantity, the candidate transformationmode is a latitudinal rectangular block transformation and a squareblock transformation; if a quantity of pixels that are among thepredicted pixels in the prediction mode and are located at a leftboundary and a lower left boundary of the encoding unit exceeds the setquantity, the candidate transformation mode is a longitudinalrectangular block transformation and the square block transformation;otherwise, the candidate transformation mode is the latitudinalrectangular block transformation, the longitudinal rectangular blocktransformation, and the square block transformation.
 9. The methodaccording to claim 2, wherein selecting the transformation modecomprises: performing a rate distortion optimization method; orperforming a simplified encoding cost comparison method to calculateonly encoding bits or calculate only distortion; or performing anagreement between an encoding end and a decoding end.
 10. The methodaccording to claim 2, wherein encoding the index of the optimaltransformation mode in the code stream comprises: when the candidatetransformation mode set comprises a latitudinal rectangular block and asquare block, indicating, in the code stream, whether the optimaltransformation mode is the latitudinal rectangular block or alongitudinal rectangular block based on a first flag bit; when thecandidate transformation mode set comprises a longitudinal rectangularblock and a square block, indicating, in the code stream, whether theoptimal transformation mode is a latitudinal rectangular block or thelongitudinal rectangular block based on a first flag bit; and when thecandidate transformation mode set comprises a latitudinal rectangularblock, a longitudinal rectangular block, and a square block, indicating,in the code stream, whether the optimal transformation mode is thesquare block transformation based on a first flag bit; and if the firstflag bit is no, indicating, in the code stream, whether the optimaltransformation mode is the latitudinal rectangular block or thelongitudinal rectangular block based on a second flag bit.
 11. Themethod according to claim 3, wherein writing the determination result inthe code stream comprises: indicating, in the code stream, whether theoptimal transformation mode is a transformation mode having thecorrespondence with the prediction mode based on a third flag bit; andif the third flag bit is no, indicating, in the code stream, that theoptimal transformation mode is one of two remaining transformation modesin the candidate transformation mode set based on a fourth flag bit. 12.A transformation mode decoding method, comprising: obtaining aprediction mode of a predicted data block that corresponds to atransformation coefficient block of an image to be decoded; selecting atransformation mode corresponding to the prediction mode from acandidate transformation mode set; and performing an inversetransformation operation on the transformation coefficient block of theimage based on the transformation mode.
 13. A transformation modedecoding method, comprising: obtaining a prediction mode of a predicteddata block that corresponds to a transformation coefficient block of animage to be decoded; determining a simplified candidate transformationmode set according to the prediction mode and a correspondence between acandidate transformation mode set and the prediction mode; parsing acode stream to obtain an index of an optimal transformation mode; andperforming an inverse transformation operation on the transformationcoefficient block of the image based on the optimal transformation modecorresponding to the index.
 14. A transformation mode decoding method,comprising: obtaining a prediction mode of a predicted data block thatcorresponds to a transformation coefficient block of an image to bedecoded; parsing a code stream to obtain a determination result aboutwhether an optimal transformation mode is a mode having a correspondencewith the prediction mode; if the determination result is no, parsing thecode stream to obtain an index of the optimal transformation mode; andperforming an inverse transformation operation on the transformationcoefficient block of the image based on the optimal transformation modecorresponding to the index.
 15. The method according to claim 12,wherein the prediction mode of the predicted data block that correspondsto the transformation coefficient block of the image comprises: anintra-frame prediction mode in intra-frame prediction; or an inter-frameprediction unit division mode in inter-frame prediction; or anintra-frame prediction unit division mode in intra-frame prediction; ora transformation mode of the transformation coefficient block of theimage in a time domain or a spatial domain of the transformationcoefficient block of the image.
 16. The method according to claim 12,wherein the candidate transformation mode set comprises: a square blocktransformation, a latitudinal rectangular block transformation, and alongitudinal rectangular block transformation; or a transformationreflecting no texture and an irregular texture, a transformationreflecting a latitudinal texture, and a transformation reflecting alongitudinal texture; or a transformation reflecting no texture, atransformation reflecting an irregular texture, a transformationreflecting a latitudinal texture, and a transformation reflecting alongitudinal texture.
 17. The method according to claim 12, whereinselecting the transformation mode comprises: if the prediction mode isan intra-frame prediction mode, when a quantity of pixels that are amongpredicted pixels in the prediction mode and are located at an upperboundary and an upper right boundary of an encoding unit exceeds a setquantity, selecting a latitudinal rectangular block transformation as atransformation type corresponding to the prediction mode; when aquantity of pixels that are among the predicted pixels in the predictionmode and are located at a left boundary and a lower left boundary of theencoding unit exceeds the set quantity, selecting a longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; otherwise, selecting a square blocktransformation as a transformation type corresponding to the predictionmode; or if the prediction mode is an intra-frame prediction mode, whena quantity of pixels that are among predicted pixels in the predictionmode and are located at an upper boundary and an upper right boundary ofan encoding unit exceeds a set quantity, selecting a non-longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; when a quantity of pixels that are among thepredicted pixels in the prediction mode and are located at a leftboundary and a lower left boundary of the encoding unit exceeds the setquantity, selecting a non-latitudinal rectangular block transformationas a transformation type corresponding to the prediction mode;otherwise, selecting a preset transformation type in all transformationtypes as a transformation type corresponding to the prediction mode; orif prediction information represents a vertical texture, selecting alatitudinal rectangular block transformation as a transformation typecorresponding to the prediction mode; if the prediction informationrepresents a horizontal texture, selecting a longitudinal rectangularblock transformation as a transformation type corresponding to theprediction mode; and if the prediction information represents no textureor an irregular texture, selecting a square block transformation as atransformation type corresponding to the prediction mode; or ifprediction information represents a vertical texture, selecting anon-longitudinal rectangular block transformation as a transformationtype corresponding to the prediction mode; if the prediction informationrepresents a horizontal texture, selecting a non-latitudinal rectangularblock transformation as a transformation type corresponding to theprediction mode; and if the prediction information represents no textureor an irregular texture, corresponding to all types of transformations;or if prediction information represents a latitudinal ornon-longitudinal rectangular block transformation, selecting thelatitudinal rectangular block transformation as a transformation typecorresponding to the prediction mode; if the prediction informationrepresents a longitudinal or non-latitudinal rectangular blocktransformation, selecting the longitudinal rectangular blocktransformation as a transformation type corresponding to the predictionmode; and if the prediction information represents a square blocktransformation, selecting the square block transformation as atransformation type corresponding to the prediction mode; or ifprediction information represents a latitudinal or non-longitudinalrectangular block transformation, selecting the non-longitudinalrectangular block transformation as a transformation type correspondingto the prediction mode; if the prediction information represents alongitudinal or non-latitudinal rectangular block transformation,selecting the non-latitudinal rectangular block transformation as atransformation type corresponding to the prediction mode; and if theprediction information represents a square block transformation,corresponding to all types of transformations.
 18. The method accordingto claim 13, wherein the simplified candidate transformation mode set isa set of candidate transformation modes having the correspondence withthe prediction mode.
 19. The method according to claim 18, furthercomprising simplifying the candidate transformation mode set, whereinsimplifying the candidate transformation mode set comprises: if aquantity of pixels that are among predicted pixels in the predictionmode and are located at an upper boundary and an upper right boundary ofan encoding unit exceeds a set quantity, the candidate transformationmode is a latitudinal rectangular block transformation and a squareblock transformation; if a quantity of pixels that are among thepredicted pixels in the prediction mode and are located at a leftboundary and a lower left boundary of the encoding unit exceeds the setquantity, the candidate transformation mode is a longitudinalrectangular block transformation and the square block transformation;otherwise, the candidate transformation mode is the square blocktransformation; or if a quantity of pixels that are among predictedpixels in the prediction mode and are located at an upper boundary andan upper right boundary of an encoding unit exceeds a set quantity, thecandidate transformation mode is a latitudinal rectangular blocktransformation and a square block transformation; if a quantity ofpixels that are among the predicted pixels in the prediction mode andare located at a left boundary and a lower left boundary of the encodingunit exceeds the set quantity, the candidate transformation mode is alongitudinal rectangular block transformation and a square blocktransformation; otherwise, the candidate transformation mode is alatitudinal rectangular block transformation, a longitudinal rectangularblock transformation, and a square block transformation.
 20. The methodaccording to claim 13, wherein parsing the code stream to obtain theindex of the optimal transformation mode comprises: when the candidatetransformation mode set comprises a latitudinal rectangular block and asquare block, parsing the code stream, and indicating whether theoptimal transformation mode is the latitudinal rectangular block or alongitudinal rectangular block based on a fifth flag bit; when thecandidate transformation mode set comprises a longitudinal rectangularblock and a square block, parsing the code stream, and indicatingwhether the optimal transformation mode is a latitudinal rectangularblock or the longitudinal rectangular block based on a fifth flag bit;and when the candidate transformation mode set comprises a latitudinalrectangular block, a longitudinal rectangular block, and a square block,parsing the code stream, indicating whether the optimal transformationmode is the square block transformation based on a fifth flag bit; andif the fifth flag bit is no, parsing the code stream, and indicatingwhether the optimal transformation mode is the latitudinal rectangularblock or the longitudinal rectangular block based on a sixth flag bit.21. The method according to claim 14, wherein parsing the code stream toobtain the determination result comprises: parsing the code stream,indicating whether the optimal transformation mode is a transformationmode having the correspondence with the prediction mode based on aseventh flag bit; and if the seventh flag bit is no, parsing the codestream, and indicating that the optimal transformation mode is one oftwo remaining transformation modes in the candidate transformation modeset based on an eighth flag bit.
 22. A transformation mode encodingapparatus, comprising: a first obtaining unit, configured to obtain aprediction mode of a predicted data block that corresponds to a residualdata block of a video image to be encoded; a first selecting unit,configured to select a transformation mode corresponding to theprediction mode from a candidate transformation mode set; and a firstprocessing unit, configured to perform a transformation operation on theresidual data block of the video image based on the transformation mode.23. A transformation mode encoding apparatus, comprising: a secondobtaining unit, configured to obtain a prediction mode of a predicteddata block that corresponds to a residual data block of a video image tobe encoded; a second determining unit, configured to determine asimplified candidate transformation mode set according to the predictionmode and a correspondence between a candidate transformation mode setand the prediction mode; a second selecting unit, configured to select atransformation mode from the simplified candidate transformation modeset, wherein the transformation mode is used to minimize distortion ofthe video image and a reconstructed video image that is obtained by adecoding end and minimize a quantity of encoding bits required forencoding the video image; a second encoding unit, configured to encodean index of the transformation mode in a code stream; and a secondprocessing unit, configured to perform a transformation operation on theresidual data block of the video image based on the transformation mode.24. A transformation mode encoding apparatus, comprising: a thirdobtaining unit, configured to obtain a prediction mode of a predicteddata block that corresponds to a residual data block of a video image tobe encoded; a third selecting unit, configured to select atransformation mode from a candidate transformation mode set, whereinthe transformation mode is used to minimize distortion of the videoimage and a reconstructed video image that is obtained by a decoding endand minimize a quantity of encoding bits required for encoding the videoimage; a third determining unit, configured to determine whether thetransformation mode is a mode having a correspondence with theprediction mode; a third encoding unit, configured to encode thedetermination result and an index of the transformation mode in a codestream; and a third processing unit, configured to perform atransformation operation on the residual data block of the video imagebased on the transformation mode.
 25. A transformation mode decodingapparatus, comprising: a fourth obtaining unit, configured to obtain aprediction mode of a predicted data block that corresponds to atransformation coefficient block of an image to be decoded; a fourthselecting unit, configured to select a transformation mode correspondingto the prediction mode from a candidate transformation mode set; and afourth processing unit, configured to perform an inverse transformationoperation on the transformation coefficient block of the image based onthe transformation mode.
 26. A transformation mode decoding apparatus,comprising: a fifth obtaining unit, configured to obtain a predictionmode of a predicted data block that corresponds to a transformationcoefficient block of an image to be decoded; a fifth determining unit,configured to determine a simplified candidate transformation mode setaccording to the prediction mode and a correspondence between acandidate transformation mode set and the prediction mode; a fifthdecoding unit, configured to parse a code stream to obtain an index ofan optimal transformation mode; and a fifth processing unit, configuredto perform an inverse transformation operation on the transformationcoefficient block of the image based on the transformation mode.
 27. Atransformation mode decoding apparatus, comprising: a sixth obtainingunit, configured to obtain a prediction mode of a predicted data blockthat corresponds to a transformation coefficient block of an image to bedecoded; a sixth determining unit, configured to determine, in acandidate transformation mode set, a transformation mode correspondingto the prediction mode; and a sixth decoding unit, configured to parse acode stream to obtain a determination result about whether an optimaltransformation mode is a mode having a correspondence with theprediction mode, and an index of the optimal transformation mode; and asixth processing unit, configured to perform an inverse transformationoperation on the transformation coefficient block of the image based onthe transformation mode.